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The »ajor objective of the Carroll County Coapetency 
Based Teacher Certificaticn tCSTC) Project is the development of a 
systea for the recertification of teachers nith classroou experience. 
A aajor goal is the development of a uodel for the identification and 
neasureuent cz teacher ccapetency areas; and the assessment of the 
extent to vhich these teacher behaviors (coupetencies) affect student 
outcoaes, and the extent to vhich they relate to school goals and 
objectives. The certification process involves several stages 
including a screening procedure^ classroom observations by trained 
observers in natural settings^ and a study of certain relationships 
outside the classroom. Collection of student outcome data is used 
during the developmental stages, not as a basis for individual 
teacher certification. The overall plan involves several steps 
including (a) establishment of a competency list and related teacher 
behaviors and student catcomes, (b) selection of appropriate 
classroom behavior and student growth measures, (c) data collection 
in on-going c'^.assrooms, (d) reduction of observation data to 
competency-related scores, and (e) examining relationships between 
observed competencies and student growth. Eeplicaticn of these steps 
successfully in additional classrooms will provide a validation of 
the findings and confirmation that some competencies were measurable 
and resulted in student growth. This in turn would provide a set of 
measurable teacher competencies that could be used as part of the 
certification process. (2C) 
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ISmKODIXTIOQfi 

Eacoura^ei! by tb* federal support for the "Title** profraw wHlch 
proliferated during the «id-sixties, researchers have had the resources 
CO Investisate a broad spectnm of educational prograMS. This federal 
largess, however, was not without its drawbacks. Progran guidelines, 
funding cycles and other considerations often proved so restrictive or 
unreasonable that the conclusions fron such programs were insignificant, 
not rcplicative, or even worthless. 3lor were the initiators of aany of 
the studies, however, without fault- Bloo« (1966) in his AERA presi- 
dential address contended that, while a 2,000 precent increase in federal 
funding for educational It 4 D provided a strong Mtive for studies of 
education, only one in 1,000 studies reported— 3 per year— were crucial 
or significant. Consequently, while the era has been an exciting and 
productive one for research, it has often been frustrating as well. 

Progras evaluators and statistical and desigr» consultants working 
with funded prcgraas arc often called upon aid--way into the studies - 
They are handicapped by insufficient prior planning, inadequate liter- 
ature review, improper instruct selection, inadequate co^arison 
groups, unreasonably smII saaples, and unrealistically short periods in 
which to test for treatment effect. Finally, the need to interrupt the 
study at crucial times to produce interim and final reports is 
especially trying— since the reports are often due before the final and 
all ia^rtant data can be collected and analyzed. 



Hie above is presented ia c^mtrast t^^Ha-«nviroi»ent In fiiiich 
the present study has been conducted. 

The Carroll Comty Coopetency Sased Teacher Certification (OTC) 
Project, conducted hy the School of Education, West Ceor^U College, 
and Carroll County, Georgia, teachers, under an ESEA Title III Crant 
f r-» the Division of Fro«ra« and Staff DevelopoMt, Georgia State 
Departsent of Education, is now in its third year of operation. The 
ujor objective of the project is the development of a systea for the 
re-certification of experienced <Jassroo« teachers which is a potential 
alternative to the progran—approved approach currently in use in 
Georgia. 

Ihe CBTC project has been characterized by a healthy freedom fro« 
nny of the problew alluded to above. Fro« the outset it has been 
proposed as a long-tera study which would be liicely to produce little 
ianediat^ results. The project design has been flexible and stri>jcct to 
imediate aodification as needed. Ihe Georgia Cepartnent of Education, 
as the grant agency has provided ainival guidelines apart f rc?i a broad 
sandate to develop a competency based certification systea, and has 
requested few forsal reports. 

This absence of unnecessary guidelines and constant interference by 
the grant agency along with a flexible design should not be viewed as a 
weakness. On the contrary, the project is established on a sound 
theoreticcl base (CBTC Librarian's Report , 1974). More inportant, it aUo 
has the advantage of regular, continuous contact with a teas of con- 
sultants, each of who« is an outstanding authority on teacher effectiveness, 
educational research or statistical analysis. Finally, this frcedo« and 
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flexibility has b«ea deliberately cultivated *e to the pratotyjiical 
nature of the project: oae hoi»ed to be worthy of replication. 

The risks of such freedo* are obvious and the investlsators are 
apprised of these risVs as w*il as the responsibility. Ibe benefits 
to be sained, however, are telt to far outwei^ the potential risks. 
The Project Design 

A iMjor goal of the project is the developMnt of a andel for the 
Identification and aeasureMnt of teacher co^tency areas; and the 
assessment of the extent to which these teacher b^viors (i.e. "com- 
petencies") affect student outco^, and the extent to which they relate 
to school goals and objectives- Although the CWC project is limited 
to Carroll County, Georgia, it is anticipated that the results of the 
study will be applicable in other school districts both within and 

outside the State. 

The project design is based on the following theoretical fraaeuork: 
The certification process will involve several stages including a 
screening procedure, classrooa observations by trained observers in 
natural settings, and a study of certain relationships outside the 
classroon. The collection of data relating to student outcoMS will be 
used during the developMnCal stages, but not as a basis for individual 
teacher certification. 

The overall plan which was established and followed during the past 
three years involved several steps including a) the establishment of a 
co^>etency list and related teacher behaviors and student outcones; 
b) the selection of appropriate measures of classroom behavior and 
student growth; c) the collection of data in on-gcing classrooms; 
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d) the reduction of cbsexvation data to co3^>«tency-relat€d scores; 

and e) exanSninjC anclci|>ated relationships betvc^en obser%ed coopetencies 

and student grovth. 

Subsequent to the successful conpletion of these preliminary steps » 
replication in additional classrooM tiould provide a validation of the 
findinirs and confiraation that som ccspetencie^ were indeed aieasurable 
and their presence resulted in student srovth. This in turn vould pro- 
vide the Department of Education with a preliminary set of measurable 
teacher behaviors which had been shown to result in student growth and 
which could then be used as part of the certification process* 

METHOD 

Introduction 

The first year of the project (1973-74) was devoted to a review of 
the literature, site visits by project staff to similar projects and 
other developfiental activities. Local teachers and administrators, 
assisted by project staff and consultants developed a set of generic 
teacher coiq>etency areas which were based in part on existing lists, 
but considered essential to all local teachers* As each competency was 
adopted as part of the definitive set, performance criteria were also 
listed. This added specificity to the definition of the competency and 
also provided guidelines for the selection of the measuring instruments 
which will be the heart of the competency-based certification system. 
Eleven generic coq>etency ar::as and some 40 sub-competencies comprised 
the resulting list. 

G 
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Durinj; the first year also, potential Instrwents were Identified^ 
additional staff vere selected^ and teachers were recruited to asrve as 
subjects of observation. The second year (1974-75) was devoted to data^ 
collection and the present (1975-76) year {Iven to additional data 
collection along with the analysis of the initial observation data. Ibe 
remainder of the present paper is devoted to a discussion of these latter 
activities. 
Subjects 

Ihe subjects of the study are the 60 teachers who participated in 
the CBTC Project during the 1974-75 school year and were selected as 
follows: All teachers in Carroll County were contacted in the Spring of 
1974 and shown a slide/ tape about the CBTC project and asked to volunteer 
as observers or observees. Three observers were selected and sixty 
teachers agreed to serve as subjects. Letters of agreement were signed 
outlining responsibilities of both parties. Suaaer workshops were con- 
ducted to familiarize participating teachers with all phases of study 
including competencies , observation instruments and student outcome 
measures. 

These teachers , being volunteers , represented grades 1 - 12, a 
wide range of subject areas and varying levels of experience. Both 
male and female and black and white teachers are represented In the 
sa^>le* 

Student Achievement Measures 

The Scott Foresman Initial Survey Test , Form A (1ST) (Honroe, Manning, 
Wepman and Gibb, 1972) was used as the pretest (fall) measure in the first 
grade. Subtests Include Reading and Math. Since the Scott Foresman 
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Reading program is used in Carroll County prianary grades, Scott Forcsman's 
1ST is used as a placciscnt test in all first grade classes. Its validity 
and reliability laake the 1ST an appropriate Instrur^nt for use in the 
present study. 

^nie selection of an end-of-year first grade laeasure vas snade after 
several different tests were tried with a saiall saicple of students not 
in the project. The Comprehensive Test of Basic Skills , Level 3, Fone S 
(CTBS) (CTB/McCraw-Hill, 1974) was found to be the oost appropriate in 
tersns of difficulty. Students were able to respond to soae of the itecs, 
but did not reach the ceiling. (Both floor and ceiling effects are 
probleias with soac tests in the lower elementary grades). 

The CTBS includes a total Reading and total Math subtests The 
reported validity and reliability of the test are satisfactory. 

The Iowa Test of Basic Skills , Primary Battery ^ Level 7, Fona 6 
(ITBS-P) (Hieronyaus, Lindquist and others, 1972), a widely used and 
well-known achievement test was used in the second grade. 

The Iowa Tests of Basic Skills , Form 5 and 6, Levels Edition (ITBS) 
(A. N. Hieronynus and E. F. Lindquist, 1971) was used with the remainder 
of the classes in grades 3 through 8. The test includes Vocabulary, 
Language, Work Study Skills, Reading, and Math Subtests. As with the 
other measures, the test's reliability and validity are adequate. In 
addition, the State of Georgia makes use of the ITBS Battery in the 
4th and 8th grades in the state-wide testing program. 

The Tests of Academic Progress ^ Form S (TAP) (Houghton Mifflin, 
1972) are designed to measure the extent to which the objectives of a 
basic area of high school instruction have been achieved. The six 

O 
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subi-^** are: Social Studies, Composition, Science, Reading, Mathemtics 
and Literature. Tiie appropriate subject area test along vith the Reading 
Test vas used in the various high school classes. The TAP Is used in 
the eleventh grade as part of the Georgia state-wide testing program. 
Validity and reliability of the instrument are satisfactory. 
Student Self-Concept Measures 

I Feel - Me Feel (IFMF) (Yeatts and Bentlor, 1970) was selected for 
use vith students in grades 1-3- This 40-item Likert-type scale 
(using five faces which range from happy to sad rather than numbers) is 
a self-concept measure appropri^^te for use with children at this level - 

Scoring was based on locally-developed factor keys which were 
entitled: Academic, Self, Frustration, Femininity, Fun and Independence. 
Coefficient Alpha reliabilities for these scales ranged from .54 to .84 
for the CBTC sample. 

How I See Myself (HISM) (Gordon, 1968) was used as a measure of 
self-concept in grades four through eight (Elementary Form) and grades 
nine through twelve (Secondary Form). Both forms of the instrument were 
scored using four of Gordon's Keys entitled Teacher-School, Physical 
Appearance, Interpersonal Adequacy and Autonomy. Reliabilities for 
these scales, using CBTC data ranged from .64 to .83. 

The Junior Index of Motivation (JIM) (Frymier, 1970) was also used 
with students in grades 9 - 12. This instrument is used to assess 
students* motivation toward school. Fifty of the 80 items are scored to 
produce a single index of motivation. Frymier (pp. 60-85) reports a 
number of studies in which the validity and reliability of the instrument 
were assessed. 

ERIC 
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Classrooia Observation Instruments 

The Coping Analysis Schedule for Education al Settings (CASES), 
(Spaulding, 1970) is a category system designed to measure pupil 
socialization and consists of 19 categories of student "coping" behaviors 
which arc identified by descriptive statements such as "Aggressive 
Behavior," "Self-Directed Activity," "Observing Passively," etc. The 
CASES categories are arranged with the more active coping categories 
grouped at one end and the more passive categories at the other, but the 
numbers do not represent a scale. CASES has been developed over a ten 
year period as a result of more than 1,000 case studies in on-going 
public school classrooms and o'her educational settings. Relationships 
between CASES and student achievement have been established by McKinney 

and others (1975) . 

The Florida Classroom Climate and Control System (FLACCS) (Soar, 
Soar and Ragosta, 1971) examines the control tactics of teachers as well 
as their affective behavior. It includes items relating to the nature of 
classroom structure, teacher and student control strategies, and teacher 
and student affective behaviors, both positive and negative. In Follow 
Through Studies (Soar, 1973), data indicated that FLACCS discriminated 
significantly between programs and related to pupil growth. 

The Observation Schedule and Record , Form 5 , Verbal (OScAR 5V) 
(Medley, 1955) looks at the verbal behavior of teachers as perceived by 
students. It is based on 14 categories for teacher questions and state- 
ments and four categories for pupil— initiated utterances; by examining 
sequence it is possible to recognize some 600 transitions from one event 
to another. OScAR has been widely used for a number of years and has been 
the basis for several other instruments. 

■ f- 
Jl\J 
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The Teacher Practices Observation Record (TPOR) (Brown, 1972) 
measures the congruency of observed teacher behavior with Dewey's 
philosophy of Experimentalism. Half of the 62 items describe beha\'ior 
which reflects agreement with Experimentalism, while the other half 
reflect disagreement. All of the items reflect behavior which is found 
in public school settings, _ 
Other Measures 

Socioeconomic Status . An estimate of the socioeconomic status (SES) 
was obtained from each student based upon the occupation of the father 
(or the mother if she were the family's sole support). This occupati'-nal 
information was transformed to a 1-8 scale (l==high status, 8=low status) 
using a modified version of Warner, Meeker and Eells (1960) classification 
of occupations and levels. As an estimate of the reliability of the 
assignment of SES, two raters independently assigned SES scores to a 
small (N=77) sample of students using the procedure described above. 
The resulting intra-class R was .94. Students with missing SES were 
assigned the mean for their classroom, rounded to the nearest integer. 

Interview. A fifty-five question interview, developed with the 
assistance of Selection Research, Incorporated, of Lincoln, Nebraska,, 
was designed to be used as a potentially useful screening instrument. 
The questions are scored dichotomously by comparing each response with a 
criterion. Responses which paraphrase the criterion are scored as 
correct and those which do not reflect the criterion are not counted. 
Items are grouped into eleven sets of five corresponding to the eleven 
CBTC competency areas. 
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Data Collection 

Student Outcomes . Achievement and self-concept measures were 
administered early in the Fall of 1974 in the classrooms of 60 Carroll 
County teachers, and posttests administered again in the classrooms of 
the same 60 teachers — all of whom were still in the project in May,- 1975. 

A group of 55pecially — trained testers administered the tests in 
each classroom. Where necessary (i.e. in primary grades) the tests were 
administered to small groups of students and care was taken to insure 
that students understood the testing procedure, instructions, etc. 

After the pretests had been administered, each of the approximately 
1800 students in the 60 classes was observed using the CASES instrument. 
This task was completed in three weeks. Each child's record was scored 
to identify his coping style. At the end of the year, two observations 
were again recorded on every student in the 60 classrooms. 

Process data . After the pretests had been administered and the 
CASES scored, six children with different predominant coping styles 
were identified in each classroom. These six students were used in 
subsequent observations using STARS. Six such observations were made 
in each classroom. 

During the year, each teacher was visited three times, and three 
five-minute samples of behavior were recorded on each occasion on the 
TPOR. Each teacher was also visited on six other occasions on each of 
which the first five minutes of behavior were recorded on FLACCS, tlie 
next ten minutes on OScAR and the last five minutes on FLACCS. 

Visits were scheduled in advance, but no attempt was made to pre- 
select the activities to be observed, since the intent nas to record a 
representative sample of the behavior present in each classroom. 
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Daring the Fall, 1974 every tcadjcr vas interviewed vith the C3TC 
interview schedule and has been interviewed again during the Fall, 1975. 
Data Reduction 

Initial Processing , Tests vere hand or znachine scored, keypunched 
and verified. Observation data vere keypunched frcjn the data collection 
fonss and verified. Achieverocnt tests vere scored according to the keys 
provided by the various p?i>lishers- The Lel£ concept sseasures vere 
reduced to subtest scores by suiming va^^lous conbin* *as of items - 
CASES frequencies for each student vere scor. I using Spaulding's procedure 
(personal coznnmnication, 197<() to identify predominant coping styles for 
each student from Style A (aggressive, amanipulative) to Style H (other- 
directed, task-oriented), as veil as a foirrposite Score (styles A -H). 

When CASES observations vere made, observers distinguished tvo types 

of classroom settings: Teacher Directed (TD) and Progran Directed (PD). 

There is evidence fron Spaulding's studies and from preliminary CBTC 

Project analyses that changes in the ciassroo^d setting resulted in changes 

in the student's CASES scores. Scores vere therefore obtained for 
i 

students for the appropriate setting by combining data from both initial 
visits only vhen collected in the same setting. The same procedure vas 
followed for the CASES data collected in final (post) visits. Ihus 
for one student, one or two sets of pre-CASES scores could be obtained 
(e.g.: both observations occurred in XD Setting; both occurred in PD 
Setting; one occurred in PD Setting and one occurred in TD Setting), and 
one or two sets of post-CASES scores obtained- 

In general, the following steps were taken to reduce the observational 
data on teachers. 
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STAPS . The 475 ceils in the student/ teacher interaction anatris 
vere weighted and a^'ersjjed to obtain an overall score and portions of 
the ratrix are coa&ined to oroduce scores reflecting various teacher 
types. Ihesc 12 scores are listed in Table 1- One set of scores was 
obtained for each student obser\'ed during each observation period and 
for the overall class if the teacher interacted vith students "t 
specifically observed. These indlvidiial scores vere averaged for all 
observations to provide class aoeans to be used in subsequent analyses. 
Ihe 475 cells were aJso retained as total fre*3ucncies across ail obser- 
vations for each teacher. 

OScAR . The record of sequential teacher-student behaviors was 
scored to produce a 600 cell anatrix of occurrences of interest (out of 
9999 possible). These vere weighted and reduced to percents to produce 
22 keys, twelve of which relate to tlie Carroll County Conpetency list. 
Ihesc are also shown in Table 1. One set of 22 scores was obtained for 
each classroom observation period, and average scores for the 6 obser- 
vation periods were then produced. Ihe 600 cell jnatrix of occurrences 
for all observations was also retained for each teacher. 

FLACCS and TPOR . These data were reduced from a series of iters 
(FLACCS=186, TP0R=62) for six series of observations to competency scores 
for each teacher by Robert and Ruth Soar at the University of Florida. 
These conpetency scores are based on a priori competencies identified 
in the Carroll County CBTC Project. They were normalized for the total 
project group (N = 60) by converting items to T scores. By combining 
sets of these items, FLACCS yielded 28 scores and TPOR resulted in 21 
scores- These are shown in Table 1- The 186 FLACCS item T scores and 
62 TPOR T scores were also retained for each teachei 
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TASLE 1 

SCBSCCS2S f©R STABS, OScAH, 
JLAOCS, IFOa Alfl) ISSTESNlEia 



VARIABLE 


KEY 


DESCai?riO!3 






STASS: 


1 


SI 


Overall coayositc 


2 


S2 


The Story Teller 


3 


S3 


The Boring Lecturer 




S4 


The Sxasiner 


5 


S5 


The Entertainer 




S6 


Hie Controller 


7 


S7 


Hie Counselor 




S8 


Ihc Psuedo— Peer 


9 


S9 


The DLscovery Teacher 


10 


SIO 


Ihe Socratic Teacher 


11 


Sil 


Hie Effective Manager 


12 


S12 


Ihe Effective E3q>ository Teacher 






OScAH: 


13 


01 


Deviant Behavior 


14 


02 


Listening 


15 


03 


Pupil Involveancnt in Subject Matter 


16 


04 


Pupil Involvement in Planning 


17 


05 


Clear Instructions 


AO 


06 


Pause for Student to Ans^jer Questions after Instruction 




07 


Clear Explanations 




08 


Pause for Qu(>d:ions aftei Explanations 




09 


Pupil Speaks Freely 


22 


010 


Self-control 


23 


oil 


Pupil Initiations 


24 


012 


Question Difficulty 


25 


013 


Teacher Statements 


26 


014 


Confidence 


27 


015 


Feedback from Teacher to Students 


28 


016 


Rebuking Behavior 


29 


017 


^Sanaging Behavior 


30 


018 


Lecturing 


31 


019 


Questions Source Modified 


32 


020 


Question Difficulty Modified 


33 


021 


Question Quality 


34 


022 


Peraissivencss of Teachers 
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TABLE 1 (continued) 



VARIABLE 



KEY 



35 Fi 

36 r2 

37 F3 

38 FA 

39 F5 
AO F6 

41 F7 

A2 F8 

A3 F9 

AA FiO 

AS Fil 

A6 F12 

A7 F13 

AS FIA 

A9 F15 

50 F16 

51 F17 

52 718 

53 ^19 
55 F20 

55 F21 

56 F22 

57 F23 

58 F2A 

59 F25 

60 F26 

61 F27 

62 F28 



FLACCS : 

Student reduced de^'iant behavior 

Student better physical, ©cntal health 

Student enjoys class, happy sislles 

Student on task, actively involved 

Student absence of vithdra'-m behavior 

Teacher p:iixses^ elicits and responds to student 
questions before proceeding 

Soseas F6 above but a sunnnative ajeasure 

Teacher uses a variety of methods, verbal, to 
deliver instructions 

Teacher uses a variety of Ksethods, verbal, plus 
selected non-verbal, to deliver instructions 

Teacher uses a variety of racthwJs — facial — gestural 

Teacher uses a variety of methods, verbal and non- 
verbal to deliver instructions 

Student self-directed to saove toward task. 

TeacJier deaonstrates proper listening skills 

Student able to speak freely 

Teacher utilizes non-verbal cozaauniiration skills 
Student able to follow directions, on task 
Students feel free to interrupt presentations 
Student snovesient toward tasks 

Teacher provides feedback to pupil on his behavior 
Teacher aiaintaias self-control in various classroom 

situations and interactions vith student 
Teacher recognizes and treats individual student 

behavior 

Teacher uses more than one instructional activity 
simultaneously 

Teacher evidence of a personal one-to-one relation- 
ship witn each student 

Student evidence of importance as class meober — group 
involveaent 

Teacher supportive classroom znanagesent 

Student evidence of enthusiasm 

Teacher listens to students and provides feedback 
Student high interest 
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TABLE 1 (continued) 



VARIABLE 

XL-K3£a KEY DESCRIFTiOS 



TFOR : 

Teacher selects goals and objectives appropriate to 
pupil need 

Teacher snatches student with appropriate snateriais 
Student on task actively involved 
Teacher gathers milti-level snateriais 
Teacher gives clear, explicit directions which are 

understood by students 
General task 

Teacher respects individual's right to speak 
Teacher helps pupil correct cognitive nisperception 
Teacher accepts necessity of dealing with individual 

students on individual basis 
Student actively involved 

Teacher — evidence of praise and/or rcis^rds in 
operation 

Teacher accepts and incorporates student ideas 
Student expresses ideas and opinions different to 

those of teacher and peers 
Teacher evidence of one-to-one counseling and 

absence of rejection (brush-off) 

Sun of Tl, T2 and T3 
Sun of Ti2 for teacher and Ti3 for S 
Teacher control — general 
Pupil choice of subject matter 
Teacher choice of subject matter 
Expansive treatsncnt of subject matter 
Restrictive treatment of subject natter 

INTERVIEW : 

Si II Diagnosis 

85 12 Organization 

86 13 Teacher connunication 

87 14 Coping 

88 15 Insightfulness 

£9 16 Student — teacher coimunication 

90 17 Methodology 

91 18 Self-concept 

92 19 Emotional awareness 

93 110 Teacher growth 

94 111 Public awareness 



63 


Tl 




T2 


65 


T3 


66 


T4 


67 


T5 


68 


T6 


69 


T7 


70 


TS 


7i 


T9 


72 


TiO 


73 


Til 


74 


Ti2 


75 


Ti3 


76 


TW 


77 


Ti5 


78 


T16 


79 


Ti7 


£0 


T18 


SI 


Ti9 


82 


T20 


83 


T21 
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a!gT£KVigg . Each interview vas Mpc recorded and the tapes vere 
transcribed. For each of the 55 questions, a correct response hzs been 
established (vhich asay be paraphrased or stated in the subject's own 
vords). Five iters en each of the eleven CBTC coirpetcncy areas are 
scrnsed to produce a stibscale score. These eleven scores, related to 
the eleven Carroll County coirpetencies, are shovn in Table 1- 

Ihe iteia-level data for STA31S (475 interaction cells), OScfiSi 
(600 occurrences), FLACCS (1S6 item T scores) and T?OR (62 item T scores) 
i;erc coE&ined into a vector of 1322 variables for each of the 60 teachers- 

Reduction of student data . The voluntary participation of teachers 
in the C3TC Project, lack of <:quivaient tests across grade levels, 
^ludent attrition, and related probleias, resulted in some significant 
corylications in the analysis of the student data. In general, the 
followin;; ^teps were taken, subs^ucnt to scoring described above for 
achievement, self-concept and CASES. 

On an instrument, by instrument basis (and vithin achievement tests, 
by subtest), in order to maintain as large a number of subjects as 
possible, student pretests and posttests vere snatched. Students «ith 
missing pretest or posttest vere dropped for that portion of the 
analysis. In some instances, primary teachers taught one group of 
students Reading and exchanged them vith another project teacher for 
Math. These students vere assigned to the "correct" teacher for each 
subtest. 

IT5S subtests for grades 3-8 vere reduced to a cocsson index by 
converting raw scores to grade equivalent scores using norms tables, and 
then subtracting students actual grade fron his grade equivalent scores. 
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tsin^ the ^JjO^V Analysis of Co^rariance pr^^raiu (Di^cn, 1974) 
.-3djustc-d postiest sieans (i.e., rcs»*^^*^^ gains) verc ccrauted for <^ach 
variable- Ir every case, pretest and SES verc «scd as covsriates to 
adjust the posttest scores. Rcsultinfi JZDCans verc transcribed and k<iy- 
pisnciJfed. Ti^e aicii5)cr of students and ocsn pretest and iscan 5ES vcre 
also recorded. 

CASES gains vere coirputed separately for ?D and for TD settinfis. 

A nalysis Procedures, 

During; the 1st year of observation and data collection, sosh* 50,000 
punched cards of rav data vere stored on disk files and a lar£e nun&er 
of variables v-ere reduced to a snaller set of indices for each of the 
60 classroeris. A cajor task during that year vas the deveioproent of 
scoring routines and other analytical processes- Eesultant statistical 
analyses vere generally descriptive of the sample. 

Due to the nature of the data collected (and supported by the 
advice of project consultants) the overall design foi; data analysis 
during the present year has been divided into three raajor phases. The 
first phase, the initial reduction of observation data to a set of 
"cocpetency scOl^s" and the coicputation of student gains, has been 
described above. Tne reduction of the observation data to coirpetency 
measures related to the CSTC competency list was undertaken in response 
CO a Ejajor project objective. The second phase is the e^anination of 
the relationships bet-jeen the competency scores and student gains and 
the third is an exanination of the observational data for patterns and 
an attecpt to sort teachers into groups. The second and third phases 
have been undertaken riore or less concurrently- 
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Cos^ctcncy scores ^nd student gains have been related ia a scries of 
saaitiplo rcgressicn Jinalyses- Sose of these ere reported else-:here at 
this AERA lacctins in p^apers by Soar and Sp-tiiuiding- Groups of teachers 
with similar cos^etency scores (identified in the Phase 3 analyses) have 
been contrasted, using student gaines as dependent variables, in a scries 
of discriminant analyses. Hie results of these analyses are discussed 
belc:^. 

Teacher groups > One major phase in the analysis of observational 
data has been an attempt to sort the teachers into groups with fairly 
hOMseneous teaching styles. Teachers with similar profiles on the 
observational laeasures fona similar clusters, or groups. To deteroine 
which teachers belong together, it is necessary to exaxnine the profiles 
of all teachers and to find those with similar profiles- The n^jrier of 
groups can range from one (i.e., all teachers exhibit similar scores on 
ijll iseasures) to N, where K=£he nunber of teachers (i.e., each teacher 
is considerably different from every other). 

Several alternative statistical raethodologies addressing this pro- 
blem of grouping are available. Transpose Factor Analysis, or 0-Analysis, 
is one of these nethods which «as selected for the initial grouping- 

The 19 X 25 (475 cell) ciatrix of STARS frequencies was sunnned 
across observations within each classrooni, resulting in a vector of 475 
"scores" for each of the 60 teachers. This total 60 x 475 riatrix was 
transposed and factor analyzed using Cuertin and Bailey's (1970) program 
ED501. Four factors were selected for rotation. These four factors 
represent an initial clustering of teachers according to STARS patterns. 
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Ihe siasscs of the teachers vho had the highest 3o2din£s on each 
reactor icere czzmlned by project staff to deterzsine if slmlarities 
existed. It vas apparent that neither grade level nor « object mtter 
vere held in coisiDon by the neizbers of an;/ group. In fact, the 
differences vere tsotc apparent than the simlarities. 

As a next step, the four groups vere contrasted on the 12 a priori 
STARS keys through a discrisainant analysis using Veldman's (1967) 
prcgrazi DISCRIM. Results indicated that the groups vere significantly 
different overall (Wilk's Lazbda = 0.079; F = 5.05, p<.0001) and 
significantly different on six of the tvelve scales. At this stage, 
the investigators felt that tvo conclusions vere supported by the data. 
First, it vas apparent that at least soae of the a priori STARS keys 
adequately described teacher characteristics and, second, and Eiore 
Icrportantly, that the presence or absence of teacher characteristics, 
or styles of teaching, could be zieasured by direct observation in live 
settings. 

The saae four groupings (from the STARS Q-analysis) vere sub- 
sequently contrasted on the 22 OScAR scores, the 28 FIACCS scores, the 
21 TFOR scores, and the 11 Interview scales through additional dis- 
crininant analyses. Each of these confirmed that the groups differed 
significantly overall on each of these jseasures (except the interview), 
and that the groups vere significantly different on several of the sub- 
scales on each instrument including 8 OScAR keys, 9 FLACCS keys, 5 TPOR 
keys and 1 Interview key. 

These results confirmed that the groups exhibited stable patterns 
^ which could be observed directly through the use of different observation 
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schedules, by diff orient observers 2nd at different tlsses. It also 
ccpfimcd that sonc of the behavioral characteristics of teachers, or 
"co35>eteacy scores" did exist in varying levels vithin the sar:plc s™"P- 
It was of jrarticular significance to the investigators that these 
characLeristics vere e^diibited by teadiers vorking in natural settings 
and vere not artificially induced by the creation of a "test" environment- 

In a subsequent snalysis^ the transposed (^75 x 60) STAK5 catrix 
vas nonsalized (i.e., converted to T-scores) and then factor analyzed. 
Four factors ^jere selected for rotation. Ihese four rotated factors 
cozprised different teachers. These groups also differed significantly 
overall and on 11 of the 12 STARS keys, 17 of 23 FLACCS keys, and 5 of 
22 OScAR keys. This suggests that multiple analyses are necessary to 
adequately identify all the characteristics of the teacher groups. 

Additional analyses have been undertaken. TWo of these include 
the transposed factor analysis of the 1322 x 60 inatrix (colusms 1-475 = 
STARS, 476-1074 = OScAR, 1075-1136 = TPOR and 1137-1132 = FLACCS) using 
raw frequencies and frequencies normalized across all subjects. 

Since Q-analysis, however, considers only the criterion of relative 
parallelness of teacher profiles in grouping and ignores overall levels 
of these profiles and relative scatter within profiles, two other 
tcethodologies arc being employed to establish groupings based on STARS, 
OScAR, FLACCS and TPOR. The relative discrimination afforded by these 
groupings are being examined by means of Bottenburg and Ward's 
hierarchical analysis, (Veldman, 1966) and Guertin's distance analysis. 
(Guertin and Bailey, 1970) 

Finally, in order to examine items in each instrument which dis- 
criminate significantly, earh of the groups identified in the various 
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Q-ujnalyses 3Te contr-asteiS on each of the 1322 itc&s using a l-i?ay ATCOVA- 
The F-ratio is not used as an inferential statistic (since snany vould 
be escpected to be significant by chance) "but rather as an index of the 
relative contribution of each v^ariablc to the overall discrisaination 
risiong the groups - 

RESLXTS 

As discussed above, trarisposed factor analysis, or Q-analysis, of 
the 60 X ^73 cell STAHS raw frequencies and the examination of the 
resulting four groups by discriminant analyses produced profiles vhich 
uere interpretable and zseaningful to the investigators- This led to 
further use of Q-analysis to identify teacher groups. A normalized 
60 >: 475 STARS ciatrix, a raw-frequency 60 x 1322 cell matrix, a 
nonnalized 60 >: 1322 cell matrix, and a standard score (z-scorO 
matrix were each subjected to Q-analysis and Varimax rotation of factors 

Following Q-analyses, the profiles of the teachers loading highest 
on the factors rotated (fron two through nine, depending on analysis) 
were contrasted by discriminant analyses- STARS, FLACCS, OScAR and TPOR 
keys which discriminated significantly among the groups helped the 
investigators to label these groups- 

The group profiles tend to be stable for the groups across instru- 
2sents, but differences appear when the teachers are 7e-grouped on the 
basis of successive analysis- This suggests both the need for multiple 
instruments in assessing teacher behavior and the need to examine 
multiple dimensions of behavior- 
Factor analysis of the transposed raw-frequency 60 x 1322 cell 
matrix and Varimax rotation of factors produced both two and three 



22 

interprccable groups of tc-icisers- Tiie results of discriminant ^analyses 
of the 3-"£roi3p solution -ire discussed 3S ^n illustration of the findings. 

TsblvC 2 presents the i:ilk*s l^iibdQ and F-ratios testing the 
significance of overall group differentiation cn profiles froa four 
instrur^ents. These indicate that the groups are significantly different 
on these dimensions- Croup centroids arc plotted on the tvo discrlcinant 
axes in Figure 1 to illustrate the separation of the groups, and to 
illustrate that the several instruments reveal different patterns of 
discrinjj nation. 

Univariate F-tests revealed that the three groups differed 
significantly on 6 STARS keys, two FLACCS and one OScAR-key, and nine 
T?OR keys. These are shovn, along vith group zseans in Table 3. Group 
1 teachers :ire characterized in part by relatively high scores on the 
"Counselor" (S7) , "Discovery" (S9) , and "Student expresses different 
ideas" (T13) keys. The Group 2 teachers are partly characterized by 
higher scores on "Boring Lecturer" (S3) and "Teacher respects student's 
right to speak" (T7), and lower scores on "Controller" (S6), "Listening" 
(02), "Student actively involved" (TIO). Finally, some of the 
significant characteristics of the 3rd group are higher scores on 
"Exaniner" (SA) , "Student njovenent toward task" (Fi8) , and "Teacher 
helps pupil correct cognitive raisperception" (T8) . Other differences 
can also be observed in Table 3. 

Finally, to 'determine if the different groups produce significant 
differences in student gain, gain measures were contrasted in dis- 
criminant analyses. The student gain on XTBS subtests did not differ 
significantly. Neither did self concept (HISM) nor CASES setting 1 gains. 
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TABLE 2 
RESULTS OF DISCRIMIHAJJT A!SALYSES 
COrCTRASTISG 3 CROUPS OK STARS, FLACCS. OScAR & TPOR 



IKSTBUliEvr 


«ILKS 
LAMBDA 


d£ 


F 


P < 


STARS 


0.213 


24, 92 


4.464 


O.GOOO 


FLACCS 


0.000 


56, 60 


1587.041 


0.0000 


OScAR 


O.ODl 


44, 72 


41.479 


0.0000 


TPOR 


0.044 


42, 74 


6.598 


0.0000 
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Key 
A ^ Group 1 
O- Group Z 
n= Group 3 



D 



5-5 

3.5 
3'0 



MX 
4-1 



O 



O 



-3.5 



-3.0 



5TARS 



-2.-5 



X)I ^2 .03 .01^ .06 .Ok .07 



OScAR 5V 
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Z2» 

too 
115 



n 



o 



A3 .tt .r5 

FL^ccs 



.50 

AO 
3S 

Jio 



A 



□ 



.rO JfS ,50 ^5 -iO >5 -70 

TPOR 



Fig.i Group Cenfroids ^rom D'lscrim'tnQni Anolyses- 
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TA3LE 3 

SIC:iFiCA!.T (p < .05) C'AKACTSJISTICS 
OF THSEE GEOUPS 
FK0>3 DlSCRl«i!iA!CT ArWiLYSS 



KEi* 

S3 
Si> 
S5 
S6 
S7 
S9 



I 2 



16. S 
17.3 
10.5 
i3.« 

3,3 
10.3 



32.2 
15.2 
14.9 
6.6 
i.2 
6.4 



iS.4 
25.9 
7.2 
13.7 
1.2 
7.5 



(df = 2.57) 



28.32 
7.92 
6.34 

13.71 
3.68 
4.49 



.0000 
.0013 
.0036 
.0001 
,0305 
-0152 



FIS 
F22 



255.5 
50.9 



230.9 
45.2 



264.6 
55.0 



5.06 
3.57 



.0096 
.0335 



02 



4.8 



2.9 



6.4 



4,04 



.0222 



Ti 
T2 
T3 
T7 
T8 
T9 
TIO 
Tll 
T13 



0.6 
0.5 
0.5 
16.8 
17.3, 
10,5 
13.4 
3.3 
10-3 



0.5 
0.7 
0.4 
32.2 
15.2 
14.9 
6.6 
1.2 
6.4 



0.5 
0.5 
0.7 
18.4 
25.9 
7.2 
13.7 
1.2 
7.5 



33.62 
76.61 
29.07 
28.32 
7-92 
6.34 
13.71 
3.68 
4.49 
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.0000 
.0000 
-GOOD 
.0000 
0013 
.0036 
.0001 
.0305 
.0152 
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H£n?evi*r fiaScs Sc C^£5 Settisns 3 sJacwd si^tiific^nt diff^r^ces for 
Style E (in fi*v£^r of Cr«i;? 2) ^iod fcr Style H (in f^-.*or of Crcvp i)- 

A difficulty in the surpnntton of groups xsnd subsequent coorsinatlon 
of differences in student outccrocs anay be illustr-^ted by cbe factor 
Inadinj^^s on seven rotated factors froa the analysis of the standaidized 
zsatrix- Each of tise seven factors had an app roxicately equal nuciber of 
positi^'e and negative high loadings^ An identification of the na^DCS 
of Che te^^chcrs with such leadings revealed groups vho appeared (to 
CliTC obs^ervers) i.o lie at cppcslta ends of a continuusi of bcljavior- 
Thus, each factor, or group, is in reality bi-polar and vithin group 
differences nay counteract differences betveen groups. 

SUKMARY AXD C0!CCLUSIO!CS 

The results of preliisinary analyses suggest that systeinatic 
classroom observation using low-inference sDcasures produces inter- 
pretable profiles of teachi^r behavior- Clear-cut relationships 
between teacher coi^etencies and student outco:i>es have not yet been 
established; however, the data suggest that finer distinctions cjust be 
snade aaong the groups before the corjpctency-outcoice relations snay be 
uncc*rered. 

The CBTC project has developed a prototype for the iDeasurezjcnt of 
conpetencies as veil as for the reduction of data related to teacher 
types. The observation instruments used in the study are worthy of 
retention. Each appears to identify unique characteristics of teachers, 
while at the sane tice conrplecenting and supporting the others. 

Further investigation of the sarne kind is needed using larger 
teacher groups at fewer grade levels. The reduction of N to 1 or 2 
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^T^^ps fox s^ZiC stJudert ssc^wrcis iM^e jr*e;3aSi-£fMl and v^iid analyses of 

Fi3;3ily, it is appar^sit frM tic preliminary analyses tiaat tlie 
siiost Sai^ortant relationships are prebaMy non-linear- The task of 
disco vex issj; vhich of the snany possible inter^ctioss jiuk^si; the teadier 
variables contribute to differences ia stwdeat jrevth vill be imast 
difficult. 

It is clear that the road to an eitpiricaily — founded co35>etency 
based certification system vill be a long one, filled vith obstacles and 
detours- SS^cvcr, the £\?al is one vhicli can be reachchi and the C37C 
project has taken the first steps down that road. 



2S 



Blc*i^, B- S. "^luH^vy-tive Year of Sdycalicnai S«^«rch,"* ^nierScan 
Eg!:c3rijDii3i Research Ji <3rnaly 1966, 3, 2ii-2iS. 

F.^r;e;^rch and nerelir.r^&eiat in E^ucMion^ IS JO, ^, I5t-22. 
C'irroil Coxjnty Ctpn^ieteni^* Exiscd Te.i!rfier CertificnLion ?roJec£. 

Cr-^.r<r!si^i\'^4* Tests of ^'^A*^; ^^'nter^sy, Callforni;3: 

!-:c€ra-f-HiIl, 1974. 

Di^33, J. (Ed.) 3i£gedSc.-sl CcCTuty Pr^^gr^a^ f 3rd JSd.) . Bt;rkelcy: 

FrvMcr* J. H. "Dev^iuj^rkfnt nnd V^sladaticn of a ^•otiv3tion Issdes," 
ll3C^*ry intc Practicg, 1970, 9 (i), S5-S8. 

Ccirdon, I. J. A Test Hrigiaal for the i!w I See Ky ^ielf Scale. 

CzlTLcsvlllitz Florida Educsitionai Hesenrch snd Deveiopnsent 
ixiuncil, 1968- 

Cucrtin, I?. H. & B.-2iiey, J. Jr. Introduction to Kader n_ Factor 
Analysis. Ann Arbor: Ewnrds Br^^thers, Inc., 1970- 

Hiernnymss, A. ^. , ^ Lindqaist, E. F. Tcwa Tests of Basic Skills. 
StJStcn: Kon^Jton Idiffiin Cor:pany, 1971. 

Hieronycnjs, A. K., Lindcuist, £. F. , & Others. leva Tests of Basic 
Skills . Boston: Houghton Mifflin Corgsany, 1972. 

^'cKsnncy, J. D. , Mason, J., Perkerson, K. , & Clifford M. 

"Relationship Between Classroom Behavior and Acadenic Achieve- 
ssent." Journal of Educatio nal Psychology , 1975, 67 (2), 
198-203. 

!*^ediey, D. M. r^bservation Schedu le a nd Record, Form 5, Verbal, 
^'i^sen^raphed , 19 73. 

Monroe, M., Manning, J. C-, k"epnan, J. , 6 Cibb, G. E. The Scott , 
Foresnan Initia l Surv ey Test. Clenview, Illinois: Scott, 
Foresmn and Company, 1972. 

Soar, R. S. Final Report — Follov Hiro ugh Classrooa Process Meas ure- 
rsent an d P upil C ro^th, (1970-71). Gainesville: Institute For 
bcveiops:ent of Hunan Resources, University of Florida, 1973- 



30 



ERIC 



29 



2\-r*-3x-rT:rf-st ni;:n53a 5:«v*^rces, raive«ity of Ficriaa, 1971- 

Ssn Jc^se: Author, 19/0- 

7^t^ ^S^,^S^iS-Tf^Z^^^' Soston: i^oa^Jslon 5<ilfiia, 1972- 

WJcisan* D- J. Fortran Tn^pr^x^ip^i for jj y J^eha yaeral Sca< g3 ces. 
rZei? Y^rk: Hoic, Hisic-li^iri: a»d Vinstoa, Inc., 1967. 

^'amcr, L., Kc-ekcr, M., S Eeils, K. %rJaI^£i^g A" America , 
::ew Yc^rk: K^rpcr, i960- 

Y*atis, ?. i Sc-arley, E. L., Jr. T Ff^cl;;::^^ 
A^jnr^iasal. Authors, 1971)- 



3 



ft 



31 



